Computing systems often rely on mouse and keyboard inputs to determine the appearance of digital objects that the user creates. Keyboard, mouse, and game controller inputs usually determine the behaviors of digital objects. People can draw objects and scan them into computers, allowing some freehand input, but the scanned drawings are then one single object. The user has limited opportunity for how to interact with the object, for example resizing and changing colors. A more natural method of inputting visual information would allow a richer graphical interaction with computing devices.
Tablets have been used as input devices. Typically the tablet senses the pressure of a stylus on a special surface and reproduces the drawn shapes on the screen. Tablets may also be used to modify photographs and drawings and as a pointing device similar to a mouse. In some versions, the pen rather than the drawing surface records the drawn input digitally. These pens use optic sensors and sometimes specially marked paper to sense and store drawings.
Some existing computing systems will recognize drawn objects as simple commands, as alphanumeric characters, or as matching objects in the system. These systems scan the drawing to identify it. However, the nature of these interactions is limited.
Some existing computing systems will recognize “air gestures” as simple commands, or more commonly as inputs into a computer game. The air gesture may be recognized by being observed by a camera or by the user holding a device that includes inertial sensors and then moving the device.